The Moving Target Function Problem in Multi-Agent Learning

We describe a framework that can be used to model and predict the behavior of MASs with learning agents. It uses a difference equation for calculating the progression of an agent’s error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agents’ learning abilities (such as its change rate, learning rate and retention rate) as well as relevant aspects of the MAS (such as the impact that agents have on each other). We validate the framework with experimental results using reinforcement learning agents in a market system, as well as by other experimental results gathered from the AI literature.